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Abstract. We propose a rationale for experimentally studying the intricate rela- 
tionship between the rate of information transmission and synchronization level in 
active networks, applying theoretical results recently proposed. We consider two 
non-identical coupled Chua's circuit with non-identical coupling strengths in order 
to illustrate the proceeding for experimental scenarios of very few data points com- 
ing from highly non-coherent coupled systems, such that phase synchronization 
can only be detected by methods that do not rely explicitely on the calculation of 
the phase. A relevant finding is to show that for the coupled Chua's circuit, the 
larger the level of synchronization the larger the rate of information exchanged 
between both circuits. We further validate our findings with data from numerical 
simulations, and discuss an extension to arbitrarily large active networks. 



1 Introduction 



Given an arbitrary time dependent stimulus that externally excites an active network, formed by 
elements that have some intrinsic dynamics (e.g. neurons or oscillators), how much information 
from such stimulus can be realized by measuring the time evolution of one of the elements 
of the network ? For example, in neurosciences, determining how and how much information 
flows along anatomical brain paths is an important requirement for understanding how animals 
perceive their environment, learn and behave [IlEllS]. 

Even though the approaches in Refs. [T1121[31I31151[5] have brought considerable understanding 
on how and how much information from a stimulus is transmitted in a neural network, the 
relationship between synchronization and information transmission in a neural as well as in an 
active network is still awaiting a more quantitative description. 

In order to treat this problem in a more analytical way, we proceed in the same line as in 
Refs. [TJIH], and study the information transfer in autonomous networks. However, instead of 
treating the information transfer between dynamical systems components, we treat the transfer 
of information per unit time exchanged between two elements in an autonomous chaotic active 
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network [9] . Arguably, the relationship between synchronization and information in autonomous 
chaotic networks is useful for understanding its counterpart in non-autonomous active networks. 

The purpose of the present work is to revisit some previous theoretical results and explain 
how to apply such approaches to study information transmission and synchronization from data 
coming from experiments. 

In Refs. [9], we proposed a formula [see Eq. (fTO|) ] that enables the calculation of the rate 
with which information is exchanged between two elements in a chaotic network, in terms of 
defined positive conditional Lyapunov exponents. Consider two non-identical coupled chaotic 
systems with two positive conditional exponents. A" and A^. The upper bound for the rate 
with which information is exchanged between these two elements is given by All -A-L. 

While Lyapunov exponents measure the exponential divergence of nearby trajectories in 
phase space, the conditional Lyapunov exponents measure the exponential divergence of nearby 
trajectories on a coordinate-transformed space. This transformed space (see Sec. |4]) is con- 
structed in such a way that if the elements in a network are almost completely synchronous |101 
1111112] . then one conditional exponent, A", measures the exponential divergence of trajectories 
along the synchronization manifold, and the other exponent, A''", measures the exponential di- 
vergence of trajectories along the transversal manifold. Then, the rate of information exchanged 
between two elements is the rate of information produced by the synchronous trajectories (A") 
minus the rate of information produced by the desynchronous trajectories (A"*"). Thus, this 
formula enables one to understand the relationship between information and synchronization, 
since the so defined conditional exponents are a measure of the synchronization and desynchro- 
nization between two elements in a network. 

We apply the formula proposed in Refs. 9J using an experimental perspective. We consider 
that one has only a short time series available to do the analysis and that the system is highly 
non-coherent. Under such conditions, we will show that the largest Lyapunov exponent of a 
two coupled Chua's circuit '13] can only be well estimated using a bivariate time series that 
contains information of the trajectories of both circuits. Further, we show that the second largest 
Lyapunov exponent can only be roughly estimated by using information from the characteristic 
of conditional observations performed in one circuit while the other realizes some event. These 
conditional observations, defined in Refs. [T4l[T5]. are in fact an alternative way of detecting 
phase synchronization Il61ll7j without having to actually measure the phase. Such a method is 
a necessary tool in order to study phase synchronization in non-coherent systems whose phases 
might not always be well defined, as the one considered here, the two non-identical diffusively 
coupled Chua's oscillators, with non-identical coupling strengths (Seed]). 

We start by showing how one can measure phase synchronization in this coupled circuit 
(Sec. [3]). Further, we demonstrate (Sec. 2]) that Eq. (|10p can be written in terms of the positive 
Lyapunov exponents, thus enabling the use of standard codes to study information transmis- 
sion in coupled chaotic systems. Since the amount of data points in each time series is small, 
alternative techniques to calculate the second largest Lyapunov exponent will be developed 
(Sec. |4T|) . The direct relationship between synchronization and information, one of the main 
results of this work, is detailed in Sec. O and finally, in Sec. [H we discuss how to extend our 
results to larger networks with arbitrary connecting topologies. 



2 Experimental and numerical simulation setups 
2.1 Experiment 

We consider two diffusively coupled non-identical Chua's circuits [13] as shown in Fig. [TJ Each 
oscillator is composed by a resistor i?i,8, an inductor Li^2, two capacitors Ci^s and C2,4, and 
one piecewise-linear resistance. In our notation, the first (second) index denotes an element in 
the upper (lower) circuit of Fig.[TJ The upper circuit is regarded as Si and the lower as 82- The 
piecewise-linear resistance is designed by a pair of linear amplifiers, U1-U2 (in Si) or U^-Ui (in 
^2) with an op-amp TAlfiA, for each oscillator. The resistance Rc sets the coupling strengths, 
ei^^and £2 = 1^. 
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Fig. 1. Two diffusively coupled Chua's oscillators with a power supply of ±9V, and parameters 
Li=20.5mH, roi=47.46t7, Ci=9.56nF, C2=95.9nF, i?2=3215n, i?3=21.28kr2, Rs=2U7Q, L2=20.3mH, 
ro2=46.98n, C3=9.93nF, C4=93.3nF, Rg^SlSGQ, _Rio=21.26kn, i?i2=2106r2. 



Two state variables, xi = Vci and X2 = Vcs, are monitored using two channels of a digital 
oscilloscope (Tektronix, TDS 220) at the nodes of the capacitors Ci and C3, respectively, for 
varying coupling resistance Rc- Data acquisition is made for 2500 data points at each snapshot 
by an 8-bit memory of the oscilloscope, with a time step-size Z\t=:0. 002ms. All circuit component 
values are precisely measured using a standard LCR-Q bridge (APLAB 4910). We consider 20 
data sets denoted by Fo, with o = {1, . . . , 20} representing the value of Rc- The larger o is, the 
larger Rc is. The set denoted as F21 contains data from the uncoupled circuits (e=0). 



2.2 Simulation 

To simulate the equations of motion of the circuit in Fig. [1] we use the dimensionless set of 
equations given by 

^ = nai[yi - Xi - f{xi)] + e^Tia^{xj - Xi) 

^ = n{xi -yi + Zi) (1) 
— = n(-Piy^ - jiZi) 

ClT 

where = (1,2) with j ^ i, n = l, = fgf, ai = g^, /?i = ^7^, 71 = ^^^^^ 

a2 = P2 = 72 = -^a^^, 61 = -^, and 62 = The state variables are the 

dimensionless voltages xi = X2 = yi = y2 = (at the respective capacitor 
nodes), zi — -^^^g^, and Z2 — ^^j^^^ (where Ili.L2 is the inductor current). E is the saturation 
voltage of the op- amps approximated as i? « 1. 

The parameters considered for the numerical simulations are roi=47.46, ro2=46.98, i?i=1650, 
i?2=3224, i?3=21300, i?4=21330, i?5=2153, i?6=221.6, i?7=220.6, i?8=1650, i?9=3194, i?io=21320, 
i?ii=21330,i?i2=2111, Ci=9. 56x10-9, C2=95.9xl0-^ 6*3=9.93x10-^ C4=93.3xl0-^ Li==20.5 
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X 10^^ and L2—20.3 x 10^^. Components have standard units as Ohm for resistance, Farad 
for capacitance and Henry for inductance. 

The piecewise-linear function f{xi^2) is defined as 



bl.,2^1.,2 + (&i,2 - ai,2), if Xi,2 < -1 
&i,2a;i,2 + (ai,2 - ^1,2), if a;i,2 > 1 



Domain £)_ 
Domain Dq 
Domain D+ 



(2) 



where ai,2 = ( — 



1 

R2,9 



)i?i,8 and 61,2 = (] 



)i?i.g. The piecewise linear function 



/(xi_2) has a slope a\^2 in the inner region near the equilibrium at the origin (domain Dq) and 
a slope h\^2 in the outer regions close to the two mirror symmetric equilibria of each oscillator 
(domains £)+ and D-)- 

The dimensionless variables in the time-r frame of the numerical simulations are obtained 
by rescaling the time-t frame of the experiment by r = -^(j^ ■ 



3 Phase, phase synchronization, and conditional maps 

Phase synchronization (PS) [16j is a phenomenon defined by 

\A^{Si,S2)\^\(f>i~m^2\<r, (3) 

where 4>i and 02 are the phases of two elements Si and S2, m — lu2/llIi is the angular frequency 
ratio that can be a real number [17] , and ui and uj2 are the average frequencies of oscillation 
of the elements and 52. The phase </) is a function constructed on a 2D subspace, whose 
trajectory projection has proper rotation, i.e. it rotates around a well defined center of rotation. 
The Chua's circuit, while presenting a double scroll attractor, has no proper rotation in the 
phase space, but it can have proper rotation in the velocity space, therefore it can admit a 
phase that measures the displacement of the tangent vector p]^[T5] and can be calculated as 
shown in Ref. [18] by 

However, as neither the simulated nor the experimental circuit, for e ^ 0, present proper 
rotation in both phase and velocity spaces, Eq. ^ has only physical meaning for a time interval 
where the attractors are far away from the equilibrium points, a time that can be large but 
not infinitely large. Therefore, for the present study is necessary to employ alternative methods 
that detect phase synchronization without having to measure the phase, as the one proposed in 
Refs. [HlfTS] , If PS exists between two subspaces, then by observing the trajectory of one circuit 
at the time the other circuit makes a physical event (an event that has positive probability of 
occurrence), there exists at least one special curve, F, in this subspace, for which the points 
obtained from these conditional observations do not visit its neighborhood. Such a curve 7^ is 
defined in the following way. Given a point xq in the attractor projected onto the subspace 
of one circuit where the phase is defined, F is the union of all points for which the phase, 
calculated from this initial point xq, reaches n{r), with n — 1, 2, 3, . . . , 00 and (r) a constant 
(typically 27r). Clearly, an infinite number of curves F can be defined. 

For coupled systems with sufficiently close parameters that have proper rotation in some 
subspace, if the points obtained from the conditional observations do not visit the whole at- 
tractor projection on this subspace, one can always find a curve F that is far away from the 
conditional observations. Therefore, for such cases, to state the existence of PS one just has to 
check if the conditional observations are localized with respect to the attractor projection on 
the subspace where the phase is calculated. Note that the value of the angular frequency ratio, 
m, is irrelevant to state PS using these conditional mappings. Whatever m is, if there is PS, 
these mappings will be localized. 

In a general situation, where the attractor has no proper rotation either in phase or velocity 
spaces and the event is a physical event, thus, as demonstrated in Ref. |15| . PS implies the 
localization of the conditional sets. 
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Fig. 2. [Color online] Projections of the attractor [gray (green) lines] and conditional mappings [black 
filled circles]. Experimental results are shown in (A-B) and simulations in (C-D). PS happens for the 
data set F12 (A) and it is absent for the data set F18 (B). PS is observed for _Rc = 12,000 (C) and is 
absent for i?c=25,000 (D). The conditional mappings for (A) [resp. (B)] are constructed by observing 
the circuit 52 at the moment the events defined in conditions ([5]) and ^ [resp. Eq. ([7])] happen in 
Si, and the conditional mappings in (C-D) are constructed by observing the circuit 5*2 at the moment 
the events defined in conditions ([§} and ((SJ happen in Si. The straight black line (A,C) illustrates a 
surface F. 



3.1 Events 

An event is considered to be the crossing of the trajectory to a Poincare section. 

The experimental Poincare sections are defined in the 2D time-delay space, constructed 
using the coordinates {x{t), x{t) + 5), with the time-delay S = 6Zir, and they are given by 

x{t + S) = Xc and x{t) > Xc (5) 
x{t + S) = —Xc and x{t) < —Xc (6) 

with Xc=l-5, for the data sets Fl to F16 plus F21, and 

x{t + S) =Xc if x{t) > Xc (7) 

with Xc~Q, for the data sets F17 to F20. The theoretical Poincare sections are defined as 

xi{t) = Xc and yi{t) < (8) 
Xi{t) = -Xc and yi{t) < (9) 

with Xr — 2. 



3.2 Observing phase synchronization in the coupled Chua's circuit without measuring the 
phase 

In Figs. [UA,C), we show the presence of PS in the experiment and in the simulations, re- 
spectively, while in Figs. HjBjD), we show the absence of such a phenomenon. While in Figs. 
(A,C), a surface F can be defined such that the conditional observations do not visit it, i.e. the 
conditional observations are localized with respect to the attractor, in (B,D) the conditional 
observations spread all over the attractor, i.e. they are not localized. 
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4 Mutual information rate, Lyapunov and conditional exponents 

In recent publications [9] , we have shown that the mutual information rate (MIR) between two 
elements in an active chaotic network, quantifying the amount of information per unit time 
that can be realized in one element, i, by measuring another element, j, is given by the sum 
of the conditional Lyapunov exponents associated with a parallel coordinate transformation 
minus the positive conditional Lyapunov exponents associated with a transversal coordinate 
transformation. 

Assuming that every element possesses only one positive Lyapunov exponent, for every 
pair of elements, whose state variables are given by and x-,-, we can define a coordinate 

transformation x^'^- = x^+Xj and xf^ = x— x^ that produces two positive conditional exponents, 
a" and (in units of bits/unit time). The mutual information rate (MIR), denoted by Ic{t), 
between the element x.^ and Xj is bounded from above by A" — A^, and thus 

/c(x„xj) < A" -A^ (10) 

where equality certainly holds if the elements are identical and are either in complete synchrony 
or decoupled (e = 0). 

As shown in Ref. [9], if there are N=2 linearly coupled chaotic systems that produce at 
most two positive Lyapunov exponents, A^ and A^, with A^ > A^, then A" — A^ and A^ = A'^, 
since the parallel and the transversal coordinate transformations are only rotations which do 
not alter the value of the Lyapunov exponents. 

This result can be easily demonstrated for the system considered here, due to its linear 
form. Thus, we can write 

/c(x,:,xj) < A^-A^ (11) 
Making the notation x — (x![",x^) and X = (xj'-2"^, xf^), we have that 

x = Mix + ci (12) 

X = M2X + C2 (13) 
X = Mx (14) 

where Mi, M2, and M are 6x6 matrices and ci and C2 are constant terms from the piecewise- 
linear function. Matrices M2 and M are explicitly written in Appendix (Sec. [8]), while matrix 
Ml is the Jacobian of Eqs. ([1]). 

Writing Eqs. and in the variational form, and making a Taylor expansion 

(which eliminates the constant terms), the following equations arc retrieved 

Cx = Mi^x, (15) 

ex = M2CX. (16) 

While the Lyapunov exponents of Eqs. ^ are calculated from Eq. (|15p . the conditional expo- 
nents are calculated from Eq. PB)) . both using the approach in Ref. [23^. But, 

Cx = M"\M2.Mex. (17) 

Noting that M^^.M2.M is just a rotation applied to matrix M2, and since a rotation does not 
change the eigenvalues of M2, thus, the Lyapunov exponents should be equal to the conditional 
exponents. 

Assuming that we have a large active network, the theoretical approaches proposed in 
[5] remain valid whenever the coordinate transformation xj'^ = x^ + Xj and x^- = x^ — Xj 
successfully separate the two systems i and j from the whole network. Such a situation arises, 
for example, in networks of chaotic maps of the unit interval connected by a diffusive (also 
known as electrical or linear) all-to-all topology, where every element is connected to all other 
elements. These approaches were also shown to be approximately valid for chaotic networks of 
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oscillators connected by a diffusively all-to-all topology. The discussion on how to extend such 
approaches to arbitrary network topologies is given in Sec. [6l 

In order to compare our results with known quantities, we will also calculate the MIR using 
Shannon's formalism [22]. The MIR between the two circuits can be roughly estimated by 
symbolizing their trajectories and then measuring the mutual information from the Shannon 
entropy of the symbolic sequences. The mutual information between 5*1 and S2 is given by 



where H{Si) is the uncertainty about what Si has sent (entropy of the message), and H{S2\Si) 
is the uncertainty of what was sent, after observing 5*2. In order to estimate the mutual infor- 
mation between the two chaotic Chua's circuit by symbolic ways, we have to proceed with a 
non-trivial technique to encode the trajectory, which constitutes a disadvantage of such tech- 
nique to chaotic systems. We represent the time at which the n-th event happens in Sk (fc={l,2}) 
by T^, and the time interval between the n-th and the (n-f l)-th event, by 5T^. 

We encode the events using the following rule. The i-th symbol of the encoding is a "1" 
if an event is found in the time interval [iA, {i + and "0" otherwise. We choose A G 

[min (5T^), max ((5T^)] in order to maximize Ig. Each circuit produces a symbolic sequence 
that is split into small non-overlapping sequences of length 1—12. The Shannon entropy of the 
encoding symbolic sequence (in units of bits) is estimated hy H = Pp log2 Pp where Pp is the 
probability of finding one of the 2' possible symbolic sequences of length I. The term H{S2\Si) 
is calculated by H{S2\Si) =—H{S2) + H{Si;S2), with H{Si; S2) representing the joint entropy 
between both symbolic sequences for 5*1 and 82- 

Finally, the MIR (in units of bits/unit time), denoted by Is, is calculated from 



The calculation of Is by means of Eq. ([TO]) should be expected to underestimate the real 
value of the MIR. Since the Chua's circuit has two time-scales, a large sequence of sequential 
zeros in the encoding symbolic sequence should be expected to be found between two events 
(large STJ} values), leading to a reduction in the value of iJ(S'i), followed by an increase in the 
value of H{S2\Si), as there will be a large sequence of zeros happening simultaneously in the 
encoding sequence for the time intervals between two events of Si and iS'2. 

4.1 Experimental exponents 

The estimation of the Lyapunov exponents from the experimental time series data was done 
using a method recently proposed (Ref. ,.19J). The first step in the algorithm is the phase 
reconstruction, accomplished by means of the nearest neighbor embedding with different time 
delays method proposed in Refs. [20]. This method considers different time delays for every 
embedding coordinate. The embedding dimension is estimated using the false nearest neighbors 
criterion proposed in Ref. yj]. The second step of this algorithm pertains estimating local 
tangent maps by a least-squares minimization with a pseudo-inverse method. Finally, in the 
third step of this method, the exponents are derived from the usual QR decomposition with a 
modified Gram-Schmidt method. 

Due to the small number of data points and the additional fact that the coupled Chua's 
circuit has a highly non-coherent dynamics, a better estimate of the largest Lyapunov exponents 
was achieved by an attractor reconstructed from the bivariate data set {xi(t),X2(t)). 

However, even the bivariate data set is not capable of providing a second largest positive 
Lyapunov exponent, A^, which should be positive if there is not complete synchronization. So, 
in order to estimate from the experimental data sets, we assume that 



I's^H{Si)-H{S2\Si), 



(18) 




(20) 
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Fig. 3. [Color online] Simulations are shown in (A-B) and experimental results in (C-D). In (A), we 
show Ic [see Eq. and 7s [see Eq. (I19|) ]. The Lyapunov exponents of the simulated circuit, with 

which Ic is calculated, are obtained using the method of Ref. [53] and the variational equations in Eq. 
(|15|) . Complete synchronization (CS) in the generalized sense [111112) is observed for Rc < 9000 and 
PS for 9000 < Rc < 16000. In (B), we show the conditional observations realized in S2 at the moment 
Si makes its n-th event, i.e. the crossing of xi{t) with the defined Poincare sections [conditions (|8} 
and ((9])]. A periodic orbit is observed for Rc — 20,000. In (C), we show calculated as described in 
Sec. 14.11 and Ic is calculated considering that is estimated from Eq. (|20|l . CS in the generalized 
sense is observed for the data series Fo, with o < 10, and PS for Fo, with 11 < o < 14. In (D), we show 
the conditional observations realized in 52 at the moment 5*1 makes its n-th event, i.e. the crossing of 
xi{t) with the defined Poincare sections [see conditions ((Sj, ([BJ, and ([7])]. A periodic orbit is observed 
for the data set F16. 



where X2 is the value of X2{t + d) at the moment the circuit 5*1 makes its n-th event. By an 
event, we consider conditions (O and ([7]). While [max ) — min {X2)] measures the size of the 
conditional observations, [max (0:2) — min (2:2)] measures the size of the reconstructed attractor. 

Thus, if the conditional observations cover the whole attractor, A^ = A^, and thus /c=0, 
which means no information is being transmitted between both circuits, since whenever Si 
crosses the defined Poincare section, S2 can be everywhere. If there is complete synchronization 
in the generalized sense |111[12] . [max(a;2) — min(a;2)] =0 and /c=A^, meaning that whenever 
Si crosses the defined Poincare section, S2 is also about to or has just crossed that particular 
section. Therefore, the information about one circuit trajectory by observing the other circuit 
is maximal. 



5 Synchronization versus information 

In Figs. EJA-B), we show results from our numerical simulations, while in Figs. (C-D), ex- 
perimental results. In both cases, one obvious observation is that the more synchronous the 
circuits are (small Rc)i the larger the rate of information that can be measured in one circuit 
about the other circuit, being maximal when the two circuits are completely synchronous in 
the generalized sense |111[12] . When both circuits are in PS, the MIR decreases but remains 
larger than when there is no PS. 
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6 Mutual information rate in large active networks 

For large active networks with elements arbitrarily connected, an extension of Eq. pH)) is 

/c(x„x,) < max(A) -A-L, (21) 

where max (A) is the largest Lyapunov exponent of the network and A"*" is the transversal 
exponent between the elements x.^ and Xj. Making an analogy with the usual definition of 
mutual information as given by Shannon ,22j, the term max (A) provides the rate of information 
produced by the source, and the term A"*" quantifies the error in the transmission. The term 
max (A) can be calculated by a scalar signal measured from Xi, and A"*^ can be estimated either 
by the ways of Eq. ((20|) or as similarly done in Ref . [15] . 



7 Conclusions 

This work proposes a rationale to experimentally study the relationship between transmission of 
information and synchronization in active networks formed by non-identical and non-coherent 
elements. For the coupled Chua's circuit, we have shown that the larger the level of synchro- 
nization the larger the rate of information exchanged between both circuits, which implies that 
such a system is non-excitable. By non-excitability [SI we mean a system that as the coupling 
strength increases, the Kolmogorov-Sinai entropy _23j (the sum of the positive Lyapunov expo- 
nents) decreases. For such systems, the maximal mutual information rate that can be achieved, 
the so called channel capacity, happens for when complete (generalized) synchronization is 
present. 

Other relevant contributions of this work include showing that for short time series the 
largest conditional exponent (demonstrated to be identical to the largest Lyapunov exponent) 
can only be reliably estimated by using a multivariate data set, with information of both ele- 
ments being considered, and that the second largest conditional exponent (which also equals 
the second largest Lyapunov exponent) can only be reliably estimated by the conditional ob- 
servations, realized in one element when the other makes an event. Finally, we have also shown 
that for two coupled non-identical and non-coherent systems, phase synchronization can be 
detected by these conditional observations, even though for such a system phase is not well 
defined. 



8 Appendix 

Consider the notation x ~ (x^,x^) and X = (xj'-j^, x|^), such that 



X = Mx , and 
X = M2X + C2 



where 
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hiai+b2a-2T 
2 



where domain (i) is defined by xi G £>_ and X2 £ -D_, or G D_ and 3:2 G -D+; domain (ii) 
by xi G D_ and 2:2 G -Dq! domain (iii) by xi G Dq and X2 G -D_ or X2 G -D+; and domain (iv) 
by xi G Dq and 2:2 G -Do- 
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